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I report on an analysis of the alignment between the South African Grade 12 
Physical Sciences core curriculum content and the exemplar papers of2008, and 
the final examination papers of 2008 and 2009. A two-dimensional table was 
used for both the curriculum and the examination in order to calculate the Porter 
alignment index, which indicates the degree of match between the two. 
Alignment indices of 0.8 and 0.6 for Physics and Chemistry, respectively, were 
computed and remained constant for Physics, but fluctuated initially for Chemis¬ 
try before levelling off. Using the revised Bloom’s taxonomy, discrepancies were 
found in terms of cognitive levels as well as content areas in both Physics and 
Chemistry. The cognitive level Remember is under-represented in the Chemistry 
and Physics examinations, whereas the cognitive levels Understand and Apply 
were over-represented in Chemistry. It is argued that the shift to higher cognitive 
levels is in line with the reported increase in cognitive complexity of the Physical 
Sciences curriculum. The significance of the study for Physical Science teachers 
is highlighted, and the potential for further research is also indicated. 

Keywords: alignment; assessment; Chemistry; cognitive level; curriculum 

content; Physics; revised Bloom’s taxonomy 


Introduction 

South African Grade 12 learners wrote their National Senior Certificate (NSC) 
examination based on the outcomes-based education (OBE) system for the 
first time in November 2008. This was also the first time that a nationally set 
public examination was written in all subjects. Only 62.5% of all the candi¬ 
dates obtained their NSC at the first attempt, while the results of the 2009 
cohort of learners showed a decline to 60.6% (DoBE, 2010:41). The new cur¬ 
riculum is designed to embody the values, knowledge and skills envisaged in 
the constitution of the new democratic South Africa. It provides learners with 
the opportunity to perform at the maximum level of their potential and 
focuses on high levels of knowledge and skills, while promoting positive values 
and attitudes (DoE, 2008a:2). 

The introduction of OBE in South Africa was intended to redress the 
legacy of apartheid by promoting the development of skills throughout the 
school-leaving population in order to prepare South Africa’s workforce for 
participation in an increasingly competitive global economy (Le Grange, 2007: 
79). The move towards standards-based assessment practices internationally 
has been incorporated into the OBE system in South Africa, with learning 
outcomes and assessment standards being specified for each school subject. 
The adoption of a radical change of policy on the curriculum was contested 
and underwent a review in 2000. The current curriculum for Grades R to 9 



572 


Edwards 


is a revised version that has been in place since 2002 (OECD, 2008:131). The 
new curriculum for Grades 10 to 12 was implemented in 2006 and culmina¬ 
ted in the NSC examination in 2008. 

A Learning Outcome is a statement of an intended result of learning and 
teaching. It describes knowledge, skills and values that learners should 
acquire by the end of the Further Education and Training band (Grades 10 to 

12) . Assessment Standards are criteria that collectively describe what a 
learner should know and be able to demonstrate at the end of a specific grade. 
Such standards embody the knowledge, skills and values required to achieve 
the Learning Outcomes. Assessment Standards within each Learning Out¬ 
come collectively show how conceptual progression occurs from grade to grade 
(DoE, 2003:7). 

The Physical Sciences subject area has been divided into six knowledge 
areas consisting of physics and chemistry components — one of these is an 
integrated knowledge area spanning both components. Approximately forty- 
five percent (45%) of the Grade 12 learners who wrote the Physical Sciences 
NSC examination in 2008 did not achieve the required pass level (DoE, 2008a: 

13) . This figure increased to 63.2% in 2009 (DoBE, 2010:49). This presents 
a huge challenge to all science teachers in the country if we are to reach 
adequate levels of scientific literacy in South Africa. The performance of South 
Africa’s Grade 8 learners in both the 1999 and 2003 Trends in International 
Mathematics and Science Study (TIMSS) was disappointing (OECD, 2008:54). 
The TIMSS study showed that learners attained the lowest average test scores 
in both mathematics and science than all other participating countries. The 
poor performance of Grade 12 learners in science is perhaps unsurprising if 
viewed against the TIMSS study. These international comparison studies have 
played a major role in identifying critical factors impacting on student achie¬ 
vement and have contributed to some extent to the current standards-based 
science education reforms in the United States and many other countries (Liu, 
Zhang, Liang, Fulmer, Kim & Yuan, 2009:780). 

In an era of accountability and the potential for schools to receive mone¬ 
tary rewards for good achievements in mathematics and science, exit exami¬ 
nations that are not aligned to the assessment standards of the curriculum 
could have serious consequences for those schools. The learners could be 
assigned grades that are not indicative of their true abilities, or instruction 
may be misguided when inferences are drawn about the extent to which 
learners have mastered the standards when the test has not adequately 
covered the content standards (Liu etal, 2009:780). Glatthorn (1999:27) has 
argued for the reconciliation between advocates and dissenters of alignment 
to see it as a tool that should be used wisely in a time of high-stakes testing 
by making it teacher-friendly and teacher-directed. 

This study is important as South Africa is a developing country with 
disparities of educational access that learners throughout the educational 
system experience as a result of inequitable policies of the past. Mapping the 
alignment of the assessment standards of the curriculum with the assessment 
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in the Grade 12 Physical Sciences exit examination could provide examiners 
with a tool with which to look at shortcomings of the examination question 
papers or the assessment standards. Herman and Webb (2007:3) argued that 
it is only when assessment is aligned with both standards and classroom 
instruction that assessment results can provide sound information about both 
how well learners are doing and how well schools and their teachers are doing 
in helping students to attain the standards. Olson (2003:2) underscores this 
point by stating that “when a test is used to measure the achievement of cur¬ 
riculum standards, it is essential to evaluate and document both the rele¬ 
vance of a test to the standards and the extent to which it represents those 
standards. Studies of alignment measure the match, or the quality of the 
relationship, between a state’s standards and its tests. That match can be 
improved by changing the standards, the tests, or both.” 

Curriculum change in South Africa 

Changes in the educational system in South Africa have been driven by con¬ 
stitutional imperatives and were characterised by policy changes influenced 
by international perspectives and global economic trends (OECD, 2008:75). 
Curriculum policy changes were followed by an implementation phase and 
currently revisions are taking place to address problems that have arisen. 
Curriculum 2005 (C2005) was launched in 1997 and was informed by princi¬ 
ples of OBE as the foundation of the post-apartheid schools’ curriculum 
(Chisholm, 2005:193). C2005 was revised and in 2002 the Revised National 
Curriculum Statement (RNCS) became policy to be implemented in 2004, and 
culminated in the phasing in of the National Curriculum Statement (NCS) in 
Grade 12 in 2008 (OECD, 2008:81). The DoE also published content frame¬ 
works in each subject as well as work schedules and subject assessment 
guidelines in response to Grade 10 to 12 teachers’ concerns about the content 
to be taught (ibid., p. 177). Rogan (2007:457) has argued that it is not enough 
to merely publish a new curriculum and assessment standards, particularly 
in a developing country. Detailed attention must be given in terms of how 
things will unfold in practice. 

Green and Naidoo (2006:79) studied the content of the Physical Sciences 
NCS and found that: 

• It reconceptualises valid science knowledge. 

• It values the academic, utilitarian and social-reconstructionist purpose 
of science. 

• It is based on a range of competences, from the metacognitive level 
through to the simple level. 

• There is a shift to greater competence complexity, and thus to 
corresponding higher expectations for teachers and learners. 

The introduction of OBE in South Africa purportedly brought about a move 
away from norm-referenced testing to criterion-referenced testing. Less em¬ 
phasis on summative assessment practices (assessment of learning) to more 
formative assessment (assessment for learning) was envisaged. In reality the 
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final examination in Grade 12 constitutes 75% of the pass requirement in 
most subjects and thus represents a summative assessment. Baker, Freeman 
and Clayton (1991) contended that the underlying assumptions regarding the 
assessments, such as norm-referenced tests and normally distributed achie¬ 
vement, can result in misalignment with standards that are targeted for all 
students (cited by Webb, 1999:1). 

Alignment Studies 

Alignment studies are important in the context of a changed curriculum as 
it may give an indication of the reform efforts when the assessment results are 
published. Schools also use the performance results to reflect on areas that 
need improvement. If there is no alignment between what is taught as spe¬ 
cified in the curriculum and what is tested, then schools may well teach to the 
test and ignore the desired assessment standards. On the other hand if 
schools teach according to the desired assessment standards and the tests 
are not aligned, it may well give a false impression of the students’ perfor¬ 
mance relative to those tests. The negative impact on remedial action is great 
because the real cause of the problem may not be addressed. La Marca (2001: 
1) maintained that it is unlikely that perfect alignment between the test and 
content standards will occur and this is sufficient reason to have multiple 
accountability measures. In supporting alignment measures he concludes 
that “it would be a disservice to students and schools to judge achievement 
of academic expectations based on a poorly aligned system of assessment”. 

Alignment in the literature has been defined as the degree to which 
standards and assessments are in agreement and serve in conjunction with 
one another to guide the system towards students learning what they are 
expected to know and do (Webb, 1999:2). The emphasis is on the quality of 
the relationship between the two. A team of reviewers analysed the degree of 
alignment of assessments and standards in mathematics and science from 
four states in the United States using four criteria: categorical concurrence 
(or content consistency), depth-of-knowledge consistency (or cognitive de¬ 
mands), range-of-knowledge correspondence (or content coverage), and ba¬ 
lance of representation (or distribution of test items). The alignment was 
found to vary across grade levels, content areas and states without any dis- 
cernable pattern (Webb, 1999:vii). The study developed a valid and reliable 
process for analysing the alignment between standards and assessments. 

Systematic studies of alignment based on Webb’s procedures have been 
completed across many states in the United States (Bhola, Impara & Bucken- 
dahl, 2003; Olson, 2003; Porter, 2002; Porter, Smithson, Blank & Zeidner, 
2007; Webb, 1997; 2007). Essentially it involves panels of subject experts 
analysing assessment items against a matrix comprising a subject area do¬ 
main and levels of cognitive demand. Martone and Sireci (2009:1337) reviewed 
three methods for evaluating alignment — the Webb, Achieve, and Surveys of 
Enacted Curriculum (SEC) methods. They focused on the use of alignment 
methodologies to facilitate strong links between curriculum standards, in- 
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struction, and assessment. The authors conclude that all three methods start 
with the evaluation of the content cognitive complexity of standards and 
assessments with the SEC methodology including an instructional com¬ 
ponent. The latter method is useful to understand both the content and 
cognitive emphases whereas the Webb and Achieve methods help to better 
understand the breadth and range of comparison between the standards and 
assessment (ibid., p.1356). 

Porter, Smithson, Blank and Zeidner (2007:28) have shown that several 
researchers have modified Webb’s procedures to measure alignment. They 
outline a quantitative index of alignment, which is a two-dimensional lan¬ 
guage for describing content. The one dimension is topics and the other 
cognitive demand. The two content dimensions are analysed by experts in the 
subject field. The cognitive demand was categorized into five levels, ranging 
from the lowest to the highest: memorize (A), perform procedures (B), com¬ 
municate understanding (C), solve non-routine problems (D), and conjecture/ 
generalize/prove (E). An alignment index is produced by comparing the level 
of agreement in cell values of the two matrices, both of which were standar¬ 
dized by converting all cell values into proportions of the grand total (Liang & 
Yuan, 2008:1825). Porter (2002:10) also contended that content analysis tools 
provide information that may be useful in developing more powerful pro¬ 
grammes for teacher professional development. The proposed continuing 
professional development of in-service teachers in South Africa could poten¬ 
tially benefit from an analysis of alignment of the curriculum and examina¬ 
tions. Areas of over- or under-representation either way could be addressed. 

Purpose of the study 

Following the recommendations of Liang and Yuan (2008:1824) that interna¬ 
tional studies be done of different countries’ curriculum standards and 
assessment systems, in this paper I analyse the extent to which the Grade 12 
Physical Sciences exemplars of 2008 (DoE, 2008c) and the examination 
papers of 2008 (DoE, 2008d) and 2009 (DoE, 2009) are aligned with the core 
knowledge and concepts embodied in the NCS for Physical Sciences (DoE, 
2003). The DoE published exemplar papers in 2008 to give an idea of what 
standard and format learners can expect in the final examination. This 
initiative followed widespread concern that the Grade 12 learners would not 
be adequately prepared for the final examination in 2008. 

In this study the following research questions and sub-questions are 
investigated: 

(a) What is the overall alignment between the Physical Sciences core 
curriculum and the Grade 12 national Physics and Chemistry 
examination? 

i. How does the alignment between the Chemistry and Physics 
examination differ? 

ii. How does the alignment of the exemplars of 2008 differ from the 
final examination of 2008 and 2009? 

(b) How do the Chemistry and Physics examinations differ from the core 
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curriculum in terms of cognitive levels? 

(c) How do the Chemistry and Physics examinations differ from the core 
curriculum in terms of content areas? 

Method 

This study employs document analysis as a research method as it involves a 
systematic and critical examination, rather than a mere description, of an 
instructional document such as the National Curriculum Statement (IAR, 
2010:1). Document analysis is an analytical method that is used in qualitative 
research to gain an understanding of the trends and patterns that emerge 
from the data. Creswell and Plano Clark (2007:114) have argued that the 
researcher must determine the source of data that will best answer the re¬ 
search question or hypotheses by considering what types of data are possible. 
Merriam (1988) also pointed out that “documents of all types can help the 
researcher uncover meaning, develop understanding, and discover insights 
relevant to the research problem” (cited by Bowen, 2009:29). The justification 
for using document analysis also stems from its efficiency in terms of time, 
accessibility within the public domain and cost effectiveness (ibid., p.31). 

A slightly modified version of Porter’s method is used to analyse the core 
knowledge and concepts of the curriculum by employing a revised Bloom’s 
taxonomy (RBT) which includes the cognitive level categories Remember, 
Understand, Apply, Analyse, Evaluate and Create (Liang & Yuan, 2008:1826). 
To facilitate the process, keywords were used to classify items and learning 
outcomes into the different cognitive levels. This was done for the curriculum 
as well as all the question papers in Chemistry and Physics. 

Rationale for using the revised Bloom’s taxonomy 

The use of the RBT, which is a two-dimensional table, was a move away from 
the restrictive hierarchical original taxonomy. The notion of a cumulative 
hierarchy has been removed so that a student may use a higher-order cog¬ 
nitive skill without a lower-order one (Anderson, 2005:106). For example, a 
student may be applying a law (say Newton’s first law) without necessarily 
understanding the law. The cognitive complexity at a lower level may be 
greater than at a higher level. These points are emphasised by Krathwohl 
(2002:215) when he states: 

However, because the revision gives much greater weight to teacher 
usage, the requirement of a strict hierarchy has been relaxed to allow the 
categories to overlap one another. This is most clearly illustrated in the 
case of the category Understand. Because its scope has been considerably 
broadened over Comprehend in the original framework, some cognitive 
processes associated with Understand (e.g. Explaining) are more cogni¬ 
tively complex than at least one of the cognitive processes associated with 
Apply (e.g. Executing). 

Mayer (2002:226) posited the idea that the revised taxonomy is aimed at 
broadening the range of cognitive processes so that meaningful learning can 
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occur. This can be achieved by not only promoting retention of material but 
transfer as well which entails the ability to use what was learned to solve new 
problems. This is particularly relevant in Physics where the learner must solve 
problems that they have not encountered before by applying their prior 
knowledge. Nasstrom (2009:40) has also shown that the revised taxonomy is 
useful as a categorisation tool of the standards for the following reasons: 

1. It is designed for analysing and developing standards, teaching and as¬ 
sessment as well as of emphasising alignment among these main compo¬ 
nents of an educational system. 

2. It has general stated content categories which allow comparisons of stan¬ 
dards from different subjects. 

3. In a study where standards in chemistry were categorised with two differ¬ 
ent types of models, Bloom’s revised taxonomy was found to interpret the 
standards more unambiguously than a model with topics-based cate¬ 
gories. 

This particular alignment study focuses on the range of competences per 
content area within Physical Sciences using the RBT. The two-dimensional 
structure of the RBT allows for teachers to increase the cognitive complexity 
of their teaching which may lead to meaningful learning (Amer, 2006:224- 
225). 

Data collection procedures 

Analysis of core knowledge and concepts in Physical Sciences 
In South Africa the NCS for Physical Sciences (Grades 10-12) has been divi¬ 
ded into six core knowledge areas: 

• two with a chemistry focus — Systems; Change; 

• three with a physics focus — Mechanics; Waves, Sound and Light; 
Electricity and Magnetism; and 

• one with an integrated focus — Matter and Materials. (DoE, 2003:11) 

A Physical Sciences content document was published in 2006 to give depth 
to the NCS (DoE, 2006). Table 1 is an extract from this document which 
shows that column one corresponds to the NCS and column two gives the 
depth to the concepts. 

The Physical Sciences Grade 12 examination guidelines give specific 
details regarding the themes to be tested in each of the knowledge areas (DoE, 
2008b: 14). It outlines the format of the question papers in Chemistry and 
Physics, knowledge that is required from content in Grades 10 and 11 and the 
core content that will be assessed in the final examination. The latter corres¬ 
ponds to the first two columns which show the core knowledge and concepts 
in Table 1. There are 32 themes in Physics including force, momentum, pro¬ 
jectile motion, work, energy, power, Doppler Effect, electrostatics, electric cir¬ 
cuits, optical phenomena, etc. Chemistry has 20 themes including organic 
chemistry, electrochemistry, rates of reactions, chemical industries, etc. 

The themes for Physics and Chemistry resulted in a 32 x 6 table and a 20 
x 6 table, respectively, for the core knowledge and the 6 cognitive levels within 
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Table 1 An example of the Grade 12 core knowledge and concepts 


The learner is able to state, explain, interpret and evaluate scientific and 
technological knowledge and can apply it in everyday contexts. 

Constructing and Applying Scientific Knowledge 

Note: The core concepts to be learned are included under the underlined 

theme and form a coherent whole. 

Core knowledge 

Core knowledge 

Core knowledge 

Comment and 

and concepts as 

and concepts 

and concepts 

links 

in the NCS for the 

proposed for 

proposed for the 


Physical Sciences 

2008 - 2010 

full implemen¬ 
tation of the NCS. 
To be updated 
before implemen¬ 
tation. 


MECHANICS 12,5% 

Motion in two 

Learners must be 

Learners must be 


dimensions: 

able to, 

able to, 


Projectile motion 

for vertical 

for two dimen¬ 

Link to Grade 10 

represented in 

projectile motion 

sional projectile 

motion in one 

words, diagrams, 

(near the surface 

motion (near the 

dimension. 

equations and 

of the Earth if air 

surface of the 

Link to Grade 11 

graphs; 

friction is ignored) 

Earth if air 

Newton’s second 


Explain that 

friction is ignored) 

Law and the Law 


projectiles fall 

calculate the 

of Universal 


freely with 
gravitational 
acceleration ‘g’ 

horizontal and 
vertical compo¬ 
nents of a projec¬ 
tile launched at 
an angle, 0, to the 
horizontal 

Gravitation 


Source: Department of Education, 2006:83. 


the revised Bloom’s taxonomy. The themes were then collapsed into the know¬ 
ledge areas for ease of comparison with the examination later. The examina¬ 
tion does not necessarily include all the themes, but all the knowledge areas 
are covered. Liu et al. (2009:782) motivated similarly in their study that “in 
order for standardised tests to properly guide instruction, it was necessary to 
focus attention on broader topics or big ideas instead of isolated and specific 
topics”. Two coders with a combined experience of more than 30 years in 
teaching Physical Sciences independently classified the curriculum content 
using the tables to achieve an inter-coder reliability of 0.97 and 0.98 for 
Physics and Chemistry, respectively. These relatively high coefficients can be 
ascribed to the use of the keywords to classify the curriculum content. The 
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assessment statements (“the learner must be able to ...”) were unambiguous 
and straightforward to classify. The differences were resolved through discus¬ 
sion. 


Analysis of the Chemistry and Physics exemplar and examination 
In 2008 the DoE published exemplars and preparatory examination papers 
for many subjects in anticipation of the final examination in November. In this 
article the Chemistry and Physics examination papers of the exemplar of 2008 
and the final examinations of 2008 and 2009 have been analysed. Each 
examination paper is worth 150 marks and together they contribute 75% of 
the pass mark in the subject. A variety of questions including multiple-choice 
questions, one-word answers, matching items, true-false items and problem¬ 
solving questions were included. The same two coders analysed the examina¬ 
tions along with the marking memoranda using a matrix identical to the one 
used for the core curriculum. An average inter-coder reliability of 0.88 and 
0.92 for the Physics and Chemistry examination respectively was obtained. 
Differences were again resolved through discussion. 


Calculating the Porter alignment index 

The cell values in the curriculum table and examination table are standardi¬ 
sed to make them comparable, that is, converted into ratios totalling to 1 (Liu 
et al, 2009:781). The Porter alignment index is defined as: 


p=i-- 




where n is the total number of cells in the table and i refers to a specific table 
cell, ranging from 1 to n. For example, for a 3 x 4 table, there are 12 cells, 
thus n = 12. Xi refers to the ith cell of Table X (e.g. the standardized test table) 
and Yi refers to the corresponding cell (ith cell) in Table Y (e.g. the content 
standard table). Both Xi and Yi are ratios with a value from 0 to 1. The sum 
of X 1 to X n is equal to 1, so is the sum of Yj to Y n . The discrepancy between 
the ith cells of the test table and the standard table can be calculated as 


Xi-Y t 

The total absolute discrepancy is then calculated by summing the absolute 
discrepancies over all cells (Liu et al., 2009:781-782). 

The values of the Porter alignment index ranges from 0 to 1, which indi¬ 
cates no alignment or perfect alignment respectively (Liang & Yuan, 2008: 
1829). The alignment indices as well as the discrepancies between correspon¬ 
ding cells, by cognitive levels and knowledge area, were all computed using 
Microsoft Excel. Liu and Fulmer (2008:375) have also argued that Porter’s 
alignment model has two advantages over other models: (a) it adopts a 
common language to describe curriculum, instruction and assessment; and 
(b) it produces a single number as the alignment index. 
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Table 2 G rade 12 Physics curriculum — core knowledge and concepts 



Remember 

Understand 

Apply 

Analyse 

Evaluate 

Create 

Sub-total 

MECHANICS 

17 

8 

14 

0 

0 

0 

39 (27.1%) 

WAVES, SOUND & LIGHT 

12 

10 

5 

1 

0 

0 

28 (19.4%) 

ELECTRICITY & MAGNETISM 

21 

16 

18 

3 

0 

0 

58 (40.3%) 

MATTER & MATERIALS 

6 

9 

1 

2 

1 

0 

19 (13.2%) 

Sub-total 

56 

43 

38 

6 

1 

0 



(38.9%) 

(29.9%) 

(26.4%) 

(4.2%) 

(0.7%) 

(0%) 

144 (100%) 


Table 3 G rade 12 Chemistry curriculum — core knowledge and concepts 



Remember 

Understand 

Apply 

Analyse 

Evaluate 

Create 

Sub-total 

MATTER & MATERIALS 

5 

2 

1 

0 

0 

0 

8 

(10%) 

CHEMICAL CHANGE 

22 

20 

6 

0 

0 

0 

48 

(60%) 

CHEMICAL SYSTEMS 

12 

8 

1 

2 

1 

0 

24 

(30%) 

Sub-total 

39 

30 

8 

2 

1 

0 




(48.8%) 

(37.5%) 

(10%) 

(2.5%) 

(1.2%) 

(0%) 

80 

(100%) 
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Results 

A. Core knowledge and concepts in Physical Sciences 

Table 2 presents the core knowledge and concepts of the Grade 12 Physics 
curriculum according to knowledge themes and cognitive levels within the 
revised Bloom’s taxonomy. Within the cognitive level categories Remember 
constitutes the largest proportion (38.9%), followed by Understand (29.9%) 
and Apply (26.4%). The last three categories have by far the lowest percen¬ 
tages as is evident from the table. The knowledge area Electricity and Magne¬ 
tism contributes 40.3% of the total cognitive categories while Mechanics 
makes up 27.1%. The knowledge theme that spans both Physics and Che¬ 
mistry (i.e. Matter & Materials) contributes 13.2% of the total. 

Table 3 presents the core knowledge and concepts of the Grade 12 Che¬ 
mistry curriculum according to knowledge themes and cognitive levels. The 
cognitive level Remember again has the greatest emphasis (48.8%) compared 
with Understand (29.9%). Apply contributes 26.4% with the other three cog¬ 
nitive levels making insignificant contributions. 

B. Overall Porter alignment index 

The overall Porter alignment indices are shown in Table 4. The alignments 
between the Physics curriculum and the exemplar as well as the 2008 and 
2009 final examinations are statistically significant, with a mean of 0.783. 
The mean alignment index for Chemistry is lower (0.628) and not significant 
at the 0.05 level. Liu et al. (2009:781) have shown that an alignment of 0.78 
is significant at the 0.05 level. 


Table 4 Porter alignment indices for Chemistry & Physics 



Exemplar 

November 

November 



2008 

2008 

2009 

Mean 

Physics 

0.795 

0.790 

0.765 

0.783 

Chemistry 

0.525 

0.670 

0.690 

0.628 


Figure 1 shows the difference in alignment between the Chemistry and 
Physics paper and the change from the exemplar to the two final examina¬ 
tions. Physics had consistently higher alignments than Chemistry and re¬ 
mained constant from the exemplar to the final examination in November 
2009. The increase in alignment for Chemistry from the exemplar to the final 
examination in 2008 is steep. There appears to be a levelling off from 2008 to 
2009. 

C. Cognitive level differences 

Table 5 presents the mean discrepancies bycognitive level for the Physics ex¬ 
emplar and November examinations. These represent the differences between 
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Figure 1 Graph of Porter alignment indices for Chemistry and Physics 


the ratios in the curriculum table and the examination table. Negative num¬ 
bers indicate that the curriculum is under-represented. The discrepancies 
ranged from +0.8% (Understand, Apply, Analyse) to -3% (Remember) for the 
Exemplar; from +0.8% (Understand & Analyse) to -3.3% (Remember) for the 
November 2008 examination; and ranged from -0.3% (Evaluate) to -3% 
(Remember) for the November 2009 examination. The largest discrepancy is 
in the Remember category for all three examinations. There appears to be no 
overall discrepancy in terms of cognitive complexity amongst the three exami¬ 
nations. 

Table 6 presents the cognitive-level differences in Chemistry. Remember 
as a cognitive level is again under-represented (-6.6%), while the Apply level 
is over-emphasised (+2.9%). The cognitive level Evaluate is also over-empha¬ 
sised (+1.1%). 

The discrepancies within the Exemplar ranges from +0.7% (Create) to 
-8.7% (Remember); within the November 2008 examination it ranges from 
-0.3% (Analyse & Evaluate) to -6% (Remember); and it ranges from -0.3% 
(Analyse & Evaluate) to -5% (Remember) for the November 2009 examination. 
The exemplar and two final examinations across all levels only marginally 
under-represent the curriculum in terms of cognitive demand. 

Figure 2 graphically illustrates how the curriculum in Physics is under¬ 
represented in the Remember category, but over-represented in almost all the 
other categories. 
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Table 5 Mean discrepancies by cognitive level, with direction, for Physics examination 



Remember 

Understand 

Apply 

Analyse 

Evaluate 

Create 

Mean 

Exemplar 2008 

-0.030 

0.008 

0.008 

0.008 

0.010 

0.000 

0.000 

November 2008 

-0.033 

0.008 

0.020 

0.008 

-0.003 

0.000 

0.000 

November 2009 

-0.030 

-0.008 

0.023 

0.015 

-0.003 

0.000 

0.000 

Mean 

-0.031 

0.003 

0.017 

0.010 

0.002 

0.000 



Table 6 Mean discrepancies by cognitive level, with direction, for Chemistry examination 


Remember 

Understand 

Apply 

Analyse 

Evaluate 

Create 

Mean 

Exemplar 2008 

-0.087 

-0.037 

0.043 

0.023 

0.040 

0.007 

-0.002 

November 2008 

-0.060 

0.040 

0.020 

-0.003 

-0.003 

0.000 

-0.001 

November 2009 

-0.050 

0.027 

0.023 

-0.003 

-0.003 

0.000 

-0.001 

Mean 

-0.066 

0.010 

0.029 

0.006 

0.011 

0.002 









584 


Edwards 


0.030 
0.020 
0010 
0.000 
- 0.010 
- 0.020 
-0.030 
•0.040 

Figure 2 Graph of mean discrepancies by cognitive level, with direction, 
for Physics examination 


r 


„ 


|ibcr Understand Apply Analyse Evaluate 


■ exemplar 2008 
Create ■ Nov 2008 
* Nov2009 


The same scenario obtains when looking at Figure 3 which presents a 
graph of the discrepancies in Chemistry. The Remember cognitive level is 
again under-emphasised, while in most of the other categories there is an 
over-emphasis. 



- 0.100 --- . - ..- 

Figure 3 Graph of mean discrepancies by cognitive level, with direction, 
for Chemistry examination 
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D. Discrepancies by content areas 

Table 7 shows the discrepancies by content areas for the Physics examina¬ 
tions, while Figure 4 illustrates this graphically. In Mechanics the biggest 
discrepancy was +1.3% for the November 2008 examination while in the 
November 2009 examination it was marginally under-represented. The 
knowledge area Waves, Sound and Light had a discrepancy of + 1.8% in the 
November 2009 examination. Electricity and Magnetism was slightly under¬ 
represented in both the final examinations. Marginal under-representation 
also occurred in the Matter and Materials knowledge area in all three Physics 
papers. 


Table 7 Discrepancies by content areas, with direction, for Physics examination 



Mechanics 

Waves, Sound 
& Light 

Electricity & 
Magnetism 

Matter & 

Materials 

Mean 

Exemplar 2008 

-0.007 

0.000 

0.010 

-0.002 

0.003 

November 2008 

0.013 

0.002 

-0.013 

-0.002 

-0.004 

November 2009 

-0.002 

0.018 

-0.015 

-0.003 

0.000 

Mean 

0.001 

0.007 

-0.006 

-0.002 




Figure 4 Graph of discrepancies by content areas, with direction, 
for Physics examination 


Table 8 presents the discrepancies by content areas for Chemistry. In the 
knowledge area Matter and Materials there is an over-emphasis (mean = 
+4.3%) in all the examination papers. Chemical Change and Chemical 
Systems are both under-represented (mean = -2.3%) in all the Chemistry 
examination papers. Figure 5 graphically shows this under-representation for 
Chemical Change and Chemical Systems. 
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Table 8 Discrepancies by content areas, with direction, for Chemistry examination 



Matter & 

Materials 

Chemical 

Change 

Chemical 

Systems 

Mean 

Exemplar 2008 

0.03 

-0.02 

-0.02 

-0.003 

November 2008 

0.05 

-0.02 

-0.03 

0.000 

November 2009 

0.05 

-0.03 

-0.02 

0.000 

Mean 

0.043 

-0.023 

-0.023 



0.05 

0.04 

0.03 

0.02 

0.01 

0.00 

• 0.01 

• 0.02 

•0.03 




MATTER & MATERIALS C 

HEMICAIQ'ANG 

E CHEMICAL SYSTEMS 

















■ exemplar 2008 

■ Nov 2008 
* Nov 2009 


•0.04 

Figure 5 Graph of discrepancies by content areas, with direction, 
for Chemistry examination 


Discussion and implications 

The classification of the core curriculum content in this paper has been 
relatively straightforward in that verbs are used to describe what a learner 
must be able to do. By using a list of keywords in each cognitive level, it is a 
matter of classifying according to the verb. Hence, the relatively high inter¬ 
rater correlations achieved by the coders can be understood, and any differ¬ 
ences are minor and easily resolved through discussion. Classifying the ex¬ 
emplar and examination items for Physics and Chemistry is a more tedious 
process. Having experienced teachers also helped to achieve good inter-rater 
correlations. Differences of opinion do arise and so the resolution through 
discussion takes much longer. 

The Porter alignment indices for Physics and Chemistry have been shown 
to be about 0.8 and 0.6, respectively. This is similar to what Liu and Fulmer 
(2008:379) found in their study. There is little variation in Physics and an 
initial fluctuation in the index for Chemistry which then becomes constant 
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between the final examinations. The Physics alignment index is also con¬ 
sistently higher than the Chemistry alignment index. The degree of alignment 
depends on the table matrix, because it would be more difficult to find exact 
matching in the corresponding cells if the table is larger, but having more 
specific categories helps. The tables that are used for Physics and Chemistry 
have specific themes under each knowledge area, which also helped with the 
coding of items. An insignificant alignment index is not necessarily a bad 
thing if this is due to the examination including more cognitively demanding 
items. There has been a differentiated impact of testing on teaching due to the 
relative emphases of the topics and cognitive demands (Liu etal, 2009:792). 
Teachers may use the items to adopt more student-centred pedagogies or they 
will use it to drive their lessons in a teacher-centred way. 

The adoption of a two-dimensional table to calculate the Porter alignment 
index is in line with the revised Bloom’s taxonomy. This allows for a non- 
hierarchical approach so that cognitive complexity can occur across levels. 
Science concepts may be mastered at different cognitive levels, but recall 
(Remember) is essential for problem-solving since it taps into long-term 
memory. The higher-order cognitive levels promote transfer of knowledge as 
opposed to formulaic methods where learners are adept at substituting 
numbers into formulae. This study has found that the Remember category 
was under-emphasised in the exemplar and final examinations for both 
Physics and Chemistry. In the Physics final examinations of 2008 and 2009, 
Apply as a cognitive level is over-represented. This could simply be an in¬ 
dication that there is a shift away from simple recall to more demanding skills 
in Physics. The final examinations of 2008 and 2009 in Chemistry em¬ 
phasised the cognitive levels Understand and Apply. Krathwohl (2002:215) 
highlighted that some cognitive processes associated with Understand are 
more complex than those associated with Apply. Green and Naidoo (2006:79) 
also showed the shift in cognitive complexity of the Physical Sciences cur¬ 
riculum overall. There is clearly a need for Physical Science teachers to im¬ 
prove the skills of learners to handle more complex items in the examination 
if the Grade 12 final examination results are anything to go by. 

A look at the examiners’ reports (WCED, 2008a: 1-2; WCED, 2008b: 1-2) 
for the final examination in Physics and Chemistry highlights the following: 

• candidates are unable to explain phenomena by applying principles; 

• new work in the curriculum is poorly answered; 

• candidates struggle with concepts requiring higher-order reasoning to 
solve problems; 

• candidates lack basic understanding (comprehension); and 

• higher-order questions that also occurred in the old curriculum are well 
answered. 

The lack of emphasis on higher-order cognitive skills could possibly lead to 
teachers not preparing Grade 12 learners adequately for the examination. 
Where teachers appear to be familiar with the work from the old curriculum, 
the learners are doing well. However, it would seem that a lack of development 
of teachers in the new materials is hampering the progress of learners. The 
learners are also doing worse in the Chemistry paper overall and a lower 
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alignment index is one possible explanation for this. The exemplar and exami¬ 
nations in Physics maintain a balanced emphasis across the knowledge areas, 
whereas for Chemistry instances of over- and under-representation occur. 
Within the knowledge area Matter and Materials in Chemistry there is a 5% 
over-representation in the final examination. This contains a lot of new 
material in Organic Chemistry that learners struggle to understand. Perhaps 
there is also a need to develop teachers in this area as many of them do not 
have an adequate background in Organic Chemistry. 

The 2009 Grade 12 Physical Sciences results that were released in Janu¬ 
ary 2010 showed a drastic increase in the failure rate. One possible expla¬ 
nation for this is that no new exemplars for Physical Sciences were made 
available in 2009. This study has shown that on the whole the final exami¬ 
nations of 2008 and 2009, and the exemplars of 2008 were fairly consistent 
across cognitive levels and content areas. What needs to be researched is the 
issue of test familiarity, since other preparatory examinations were also 
released in 2008. Umalusi (2009:121) compared the NCS with the old curri¬ 
culum and concluded that it is far more difficult in terms of breadth of 
content than is the old curriculum, but is midway between the Higher and 
Standard Grade levels of the old curriculum in terms of the levels of difficulty 
of the content topics. The Physical Sciences panel also estimated that up to 
35% more classroom time is needed for the NCS content compared with the 
old curriculum. 

There are huge gaps inherent in the South African education system as 
far as implementing the new curriculum is concerned. Teachers utilise the 
published policy documents to give direction to their teaching. The curriculum 
documents themselves should be clear and give a balanced weighting across 
cognitive levels. If this situation obtains, then teachers can have a reasonable 
expectation that the examination should be aligned to the curriculum content. 
Alignment studies would thus be important to elucidate the differences in the 
intended curriculum and the enacted curriculum. They could also be used as 
a tool for the professional development of teachers to analyse their internally- 
set examinations. This study makes a first contribution towards establishing 
the quality of the relationship between the Physical Sciences curriculum and 
the examination in the South African context. It also has potential to be 
utilised in other subjects such as Mathematics and Life Sciences. A further 
area of research, which has been beyond the scope of this study, would be to 
examine the cognitive complexity within each knowledge area in Physics and 
Chemistry. 
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